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Abstract —To accommodate the explosive growth in mobile 
data traffic, both mobile cellular operators and mobile users 
are increasingly interested in offloading the traffic from cellular 
networks to Wi-Fi networks. However, previously proposed 
offloading schemes mainly focus on reducing the cellular data 
usage, without paying too much attention on the quality of service 
(QoS) requirements of the applications. In this paper, we study 
the Wi-Fi offloading problem with delay-tolerant applications 
under usage-based pricing. We aim to achieve a good tradeoff 
between the user’s payment and its QoS characterized by the 
file transfer deadline. We first propose a general Delay-Aware 
Wi-Fi Offloading and Network Selection (DAWN) algorithm for 
a general single-user decision scenario. We then analytically 
establish the sufficient conditions, under which the optimal policy 
exhibits a threshold structure in terms of both the time and 
file size. As a result, we propose a monotone DAWN algorithm 
that approximately solves the general offloading problem, and 
has a much lower computational complexity comparing to the 
optimal algorithm. Simulation results show that both the general 
and monotone DAWN schemes achieve a high probability of 
completing file transfer under a stringent deadline, and require 
the lowest payment under a non-stringent deadline as compared 
with three heuristic schemes. 

Index Terms —Mobile data offloading, cellular and Wi-Fi 
integration, dynamic programming, threshold policy. 


I. Introduction 

M OBILE cellular networks nowadays are often heavily 
loaded due to the huge amount of mobile data traffic 
generated, for example, through mobile web browsing and 
mobile video applications. According to Cisco’s forecast, 
mobile data traffic will increase by 11-fold between 2013 
and 2018 globally 0. On the other hand, the mobile cellular 
network capacity is growing at a much slower pace, so that it 
is likely that the mobile traffic demand will exceed the network 
capacity in the short to medium term (3). As a result, there is 
an urgent need from the mobile operators (MOs) worldwide 
to increase the network capacity in a cost-effective and timely 
manner. An efficient way to ease the cellular congestion is to 
use complementary technologies, such as Wi-Fi |4j, to offload 
the traffic originally targeted towards the cellular network. 
Juniper Research estimated that only 40% of the global mobile 
data traffic will reach the cellular network in 2017, as most of 
the traffic are likely to be offloaded using Wi-Fi 0. 
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There are two main approaches for the initiation of Wi-Fi 
offloading, namely user-initiated and operator-initiated offload¬ 
ing. In the user-initiated offloading, the mobile user (MU) 
is responsible for selecting the network technologies that it 
intends to use. In the operator-initiated offloading, however, 
the operator profile stored in the mobile device prompts the 
connection manager to initiate the offloading procedure. The 
MOs would prefer the operator-initiated offloading, as it gives 
them a better control on users’ network selections. However, 
since the operator-initiated offloading involves complicated 
network control between the MOs and the MUs, further 
standardization and development are still under way. Currently, 
the user-initiated offloading is the more popular choice due to 
its simplicity in implementation, and it will be the focus of 
this paper. 

New functionalities in some recently proposed IEEE and 
3GPP architectures can provide MUs with useful network 
information for the user-initiated offloading. In Hotspot 2.0, 
which is based on the IEEE 802.1 lu standard 0, the network 
discovery and selection functionality advertises the network 
information related to the access network type, roaming con¬ 
sortium, and venue information through management frames. 
The access network discovery and selection function (ANDSF) 
server 0, 0, proposed in 3GPP Release 11, can assist an 
MU to choose a suitable Wi-Fi network by providing it with 
a list of preferred access networks and the access network 
discovery information. Moreover, it is envisaged that network 
information, such as real-time load and radio conditions, can 
be broadcast to the MUs through the system information 
block (SIB) messages currently used in the LTE system J8] 
pp.46]. With these new architectures for cellular and Wi-Fi 
integration, MUs can make intelligent network selection and 
offloading decision based on real-time network load and price 
information. 

In addition, the standardization effort from the industry has 
been accompanied by a series of efforts on the characterization 
of Wi-Fi offloading performance from the academia. Recently, 
measurement studies 0, IH demonstrated that Wi-Fi of¬ 
floading can significant reduce the cellular network congestion. 
In fact, the potential benefit of data offloading is even more 
significant 0, El for delay-tolerant applications, such as 
e-mail, movie download, and software update, which can 
tolerate delays ranging from several minutes to several hours 
without significant negative impact on users’ satisfactions. For 
example, the survey in IflTTl reported that more than half of 
the respondents are willing to wait for 10 minutes to stream 
YouTube videos and 3-5 hours to download a file when a 
monetary incentive is given. 
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In this paper, we study the user-initiated Wi-Fi offloading 
problem for delay-tolerant applications, where a user aims 
to minimize its total data usage payment under usage-based 
pricing , while taking into account the deadline of its applica¬ 
tion. Previous works on user-initiated Wi-Fi offloading policy, 
which includes 0 , ca, im, m, mainly focus on reducing 
the cellular data usage without paying too much attention to 
the quality of service (QoS) of the user’s application. As an 
example, consider the on-the-spot offloading (OTSO) scheme 
that most smartphones are using by default fT~2l . The OTSO 
scheme adopts a simple offloading policy that an MU offloads 
its data traffic to a Wi-Fi network whenever possible. However, 
our simulation study suggests that it is not always desirable to 
offload to Wi-Fi whenever possible, especially when the Wi-Fi 
network is highly loaded and the deadline is tight. However, in 
general it is challenging to achieve a good balance between the 
total payment and the QoS when taking various factors such 
as network conditions and delay deadlines into consideration. 

First, we consider a general user offloading scenario, and 
formulate the delay-aware Wi-Fi offloading problem as a 
finite-horizon sequential decision problem. We propose a gen¬ 
eral Delay-Aware Wi-Fi Offloading and Network Selection 
(DAWN) algorithm, which achieves a good tradeoff between 
the total payment and the QoS. However, in general, a sequen¬ 
tial decision problem is computationally intractable unless the 
optimal policy has a threshold structure in. To this end, 
based on the concepts of superadditivity and subadditivity 
E3 pp. 103], we derive sufficient conditions under which 
the optimal policy exhibits threshold structures in terms of 
both time and the remaining file size to transfer. It motivates 
us to design the monotone DAWN algorithm with a much 
lower computational complexity that approximately solves the 
general offloading problem. To the best of our knowledge, 
this is the first paper that studies offloading algorithm design 
analytically, which tradeoffs a user’s payment and QoS. The 
insights obtained, even under the single-user setting in the 
user-initiated offloading, are crucial for us to understand the 
more complicated multi-user offloading problems in commer¬ 
cial networks. 

In summary, the main contributions of our work are as 
follows: 

• Optimal user-initiated offloading algorithm'. We consider 
the Wi-Fi offloading problem for delay-tolerant appli¬ 
cations, and propose a general DAWN algorithm that 
achieves a good tradeoff between total data usage pay¬ 
ment and the user’s QoS. 

• Low-complexity approximation offloading algorithm: We 
derive sufficient conditions under which the optimal pol¬ 
icy has a threshold structure, and propose a monotone 
approximation DAWN algorithm with a much lower 
computational complexity. 

• Optimal offloading decisions: Simulation results show 
that the general and monotone DAWN algorithms achieve 
a high probability of file transfer completion and require 
a low payment as compared with three heuristic schemes. 
We also show that Wi-Fi offloading may not be desirable 
under a tight deadline constraint and a congested Wi-Fi 
network. 


The rest of the paper is organized as follows. We first 
review the literature in mobile data offloading in Section HI] 
We describe our system model in Section [III] and formulate 
the delay-aware Wi-Fi offloading problem in Section [IV] We 
propose the general DAWN algorithm for the general case in 
Section |V] and the monotone DAWN algorithm for the special 
case with threshold optimal policy in Section |VT] Simulation 
results are given in Section IVIII and the paper is concluded in 
Section MU 

II. Literature Review 

The existing mobile data offloading literature focuses on 
either economics or technology issues. Related to network 
economics, Zhuo el al. in Ml considered a 3G cellular 
network, where the MO uses discount coupons to incentivize 
MUs to use delayed data offloading. The problem was for¬ 
mulated as a reverse auction with one buyer and multiple 
sellers, where the MO is the buyer, and the MUs are the 
sellers. loe-Wong et al. in m studied the user adoption 
of supplementary technology (e.g., Wi-Fi or femtocell) for 
cellular traffic offloading. The utility function of each user 
is related to its valuation of the technology, the congestion 
level, and the flat pricing of the service provider. The studies 
in m, I I3l considered an offloading market, where the MOs 
pay the third-party deployed APs for data offloading. Gao et 
al. in Ifl8l characterized the subgame perfect equilibrium in a 
data offloading game, where the base stations (BSs) propose 
the market prices, and the APs determine the volume of data 
traffic that they are willing to offload. Iosifidis et al. in El 
proposed an iterative and incentive compatible double auction 
that maximizes the social welfare. Lee et al. in l20l studied the 
economic aspects of Wi-Fi offloading in a monopolistic market 
with multiple MUs and one MO. Each MU is characterized by 
its willingness to pay, traffic demand, delay profile, and Wi-Fi 
contact probability. 

Related to the mobile data offloading technology, Dimat- 
teo et al. in ED evaluated the costs and benefits of Wi¬ 
Fi offloading in metropolitan area with real mobility traces. 
They characterized the number of Wi-Fi access points (APs) 
required for the support of a given QoS requirement. Bennis 
et al. in |]22l studied the subband selection, power allocation, 
and scheduling problem of a small cell base station, which 
can transmit with both the cellular and Wi-Fi interfaces. The 
base stations can self-organize and adjust their transmission 
strategies using reinforcement learning. There are a number 
of recent research results on the study of delayed Wi-Fi 
offloading policy. Balasubramanian et al. in 0 conducted a 
measurement study on Wi-Fi availability for moving vehicles. 
They proposed the Wiffler system for data offloading based on 
the prediction for future Wi-Fi availability using past mobility 
history. Lee et al. in HE) performed another measurement 
study on Wi-Fi offloading with pedestrians. They conducted 
trace-driven simulations to study the impact of various pa¬ 
rameters on the offloading efficiency. Ristanovic et al. in El 
considered energy-efficient offloading for delay-tolerant appli¬ 
cations. They showed that the proposed offloading algorithms 
can offload a significant amount of traffic from the cellular 
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Fig. 1. An example of the network setting, where the MU is moving within 
a set of C, = {1,..., 16} locations. The MU is always under the coverage 
of a cellular BS, but Wi-Fi is only available at four locations, where = 
{4,11,13,16}. The rest of the locations do not have Wi-Fi, i.e., = 

We assume that the MU is sending a file of size K bits that should 
be completed by deadline T. Given the mobility pattern of the MU, it aims 
to decide whether it should remain idle (a = 0), use the cellular network 
(a = 1), or use the Wi-Fi network (a = 2) if it is available in each time slot 
to reduce its payment under usage-based pricing, while taking into account 
the deadline of the application. 

network and extend the battery lifetime. Im et al. in f23l 
considered the cost-throughput-delay tradeoff in user-initiated 
Wi-Fi offloading. Given the predicted future usage and the 
availability of Wi-Fi, the proposed system decides on the 
application that should offload its traffic to Wi-Fi at a given 
time, while taking into account the cellular budget constraint 
of the MU. 

In fact, similar to this paper with a detailed user’s decision 
model, the works related to data offloading algorithm design in 
(91 , IfTOl , 1T3I , Il23l focus on the single-user offloading prob¬ 
lem. On the other hand, the works related to data offloading 
economics in GD-iia considered simplified models on users’ 
decisions, and they mainly focus on the multi-user offloading 
problem. 

III. System Model 

As shown in Fig. [Q we consider an ML0 moving within 
the coverage of the cellular network, such that the cellular 
connection is always available to the MU. Occasionally, the 
MU may be able to access Wi-Fi APs at some locations (e.g., 
in a coffee shop or in a shopping mall). In other words, the Wi¬ 
Fi connection is location-dependent and may not be available 
to the MU at all time. The MU is running a file transfer 
application, which requires transferring of K bits within T 
time slots. In other words, the file transfer application is delay- 
tolerant with a deadline T CD. For example, an MU on the 
road wants to send an e-mail with a large attachment of 20 
Mbytes through his smartphone in the next 10 minutes. The 
MU moves in a set C = {1 of possible locations, 

following a Markovian mobility model that can be derived 

1 In this paper, since we focus on the user-initiated offloading, it is 
reasonable to consider the setting that a single MU makes an independent 
decision without coordinating with the other MUs. We believe that it is an 
important step towards a better understanding of the multi-user offloading 
problem in the operator-initiated offloading, where the MO needs to decide 
on the offloading decisions of multiple MUs. 


based on the past mobility pattern of the MU. Such a model 
is widely used in the literature l24l - |26| . 

We consider the usage-based pricing used by MOs (such 
as the one used by Verizon Wireless Il27l ). where the usage 
price of the cellular network is often higher than that of the 
Wi-Fi network. It should be noted that the pricing scheme is 
general, and it includes free Wi-Fi as a special case. When 
making the offloading decisions, the MU needs to take into 
account the payments regarding different network types and 
its QoS requirement in terms of file transfer completion. First, 
the MU has the incentive to offload as much data traffic to 
the Wi-Fi network as possible, so as to reduce its payment. 
This means that the MU prefers to defer the transmission 
until a Wi-Fi hotspot is available. On the other hand, the MU 
should also consider whether it can complete the file transfer 
by the deadline. For example, if the remaining time before the 
deadline is short, then the deferred transmission may violate 
the deadline if the MU does not have enough opportunities to 
transmit through Wi-Fi in the near future. In this case, instead, 
the MU should start the file transfer using the ubiquitous 
cellular connection as soon as possible to reduce the latency. 
To sum up, an efficient delay-aware Wi-Fi offloading scheme 
needs to achieve a good tradeoff between the total data usage 
payment and the MU’s QoS requirement. 

As the Wi-Fi offloading problem involves decision making 
in multiple time slots before the deadline, we formulate it as 
a finite-horizon sequential decision problem in the following 
section. We aim to find the MU’s optimal transmission policy, 
which minimizes the MU’s data usage payment, while taking 
into account the deadline of the file transfer application. By 
defining the total cost as the total payment and a penalty for 
not finishing the file transfer by the deadline, we can derive 
the optimal transmission policy through dynamic programming 
(DP). We further propose an approximation algorithm based 
on a non-standard DP theory. 

IV. Problem Formulation 

In this section, we formulate the delay-aware Wi-Fi offload¬ 
ing problem of a single MU as a. finite-horizon sequential de¬ 
cision problem CD. Without loss of generality, we normalize 
the length of a time slot to be one. The MU needs to choose 
an action (to be explained later) at each decision epoch 

ter={i,...,T}. (l) 

The system state is defined as s = (k, l). The state element 
k 6 K, C [0 ,K] represents the remaining size (in bits) of a 
fileS to be transferred. The state element l £ C = {1,L} 
is the location index, where L is the total number of possible 
locations that the MU may reach within the T time slots. As 
shown in Fig. [Q let C C and ff 1 ' 1 C C be the sets 
of locations where Wi-Fi is not and is available, respectively, 
such that = C\d^>. 

The action a specifies the transmission decision of the 
MU at each decision epoch. Specifically, we have a £ A = 

2 For the case where the MU is transferring multiple files, we can include 
additional state elements and decisions, and solve the problem by dynamic 
programming. 
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{0,1, 2}, where a = 0 means that the MU chooses to remain 
idle, a = 1 means that the MU transmits through cellular, and 
a = 2 represents that the MU transmits through Wi-Fi. Notice 
that actions a = 0 and a = 1 are always available to the MU 
at all locations. Action a = 2, however, is only available at 
a location l £ Thus, the available choice of action a 

depends on the state element l, so a € A^ C A, where A^ 
is the set of available transmission actions at location l: 


w ({0,1,2}, if 

({0,1}, if i e £ (0) . 


( 2 ) 


We adopt the commonly used usage-based pricing, where 
the payment of an MU is directly proportional to its data usage. 
Let p(l,a) be the price per unit of usage for choosing action 
a £ A^ at location l, where p(l, 0) = 0, V( £ C for the 
idle action. It should be noted that we consider a general 
location and network dependent pricing, which includes the 
commonly used location independent pricing as a special case. 
Let p(l , a) be the estimated throughput of the user at location 
l with action a £ A^, where p(l, 0) = 0, Ml £ C when 
the MU remains idle (i.e., when a = 0). We would like 
to mention that p.(l,a) can take into account the congestion 
effect when multiple MUs are simultaneously using the same 
network]! We assume that the MU can obtain such real-time 
price and data rat^! information for accessing networks at 
different time and locations through the system information 
block (SIB) announced by the MO, as discussed in Section [0 
0. The payment of the MU at state s with action a £ A 1 ' 1 ' 
at time slot t £ T is 


c t {s, a) = c t (k , l, a) = min{fc, p{l , a)}p(l, a), (3) 

which is equal to the data usage payment in the time slot. 

After the deadline has passed, we define the penalty for not 
being able to finish the file transfer at state s as 

Ct+i(s) = c T +i(k,l) = h(k), (4) 


and [x] + = max{0, x}. p{l’ 1 1) is the probability]! that the MU 
will move from location l to location and it is estimated 
based on the past mobility pattern of the MU Il24l - li26l . 

Let St : K, x C —>• A be a function that specifies the 
transmission decision of the MU at state s = ( k,l ) and time 
slot t. We define a policy it = (S t (k,l), V k £ 1C, l £ C,t £ T) 
as the set of decision rules for states and time slots. We denote 
sf = (kf,lf) as the state at time slot t if policy 7r is used, 
and we let II be the feasible set of 7r. The MU aims to find 
an optimal policy 7T* that minimizes the sun]! of the expected 
total payment from t, = 1 to t = T and the penalty at t = T+ 1 
as follows: 


' T 

J2 Ct ( S t’ 6 t( S 7)) +CT+l(s?+l) ■ 

t = 1 

(7) 

E™ denotes the expectation with respect to the probability 
distribution of the MU mobility model and policy 7r with an 
initial state Si = (K,l i), where f is the location of the MU 
at t = 1. 


minimize E™ 
•n-en Sl 


V. General DAWN Algorithm 

In this section, we solve problem 0 optimally using finite- 
horizon DP for the general penalty function, network usage 
prices, and cellular/Wi-Fi data rates. We propose a general 
DAWN algorithm that computes the optimal policy. 

Let vt(s) be the minimal expected total cos]! of the MU 
from time slot t to T + 1, given that the system is in state s 
immediately before the decision at time slot t. The optimality 
equation E3 pp. 83] relating the minimal expected total cost 
at different states for t £ T is given by 

v t {s) =v t (k,l) = min {ip t (k,l,a)}, (8) 

where for k £ /C, l £ C, and a £ A^ l \ we have 


where h(k) > 0 is a non-decreasing function of k with h{ 0) = 
0. The subscript T + 1 means that we compute the penalty at 
the beginning of the T + 1 time slot (immediately after the 
deadline). In fact, the MU chooses h{k) according to the QoS 
requirement of its application. 

The state transition probability p(V|s,a) = 
p[(k\ l') | (k, l), a) is the probability that the system 
will go into state s' = (k 1 , l') in the next time slot if action 
a is taken at state s = (, k , l). Since the movement of the MU 
from location l to location l' is independent of the file size k 
and transmission action a, we have 


p(s' | 
where 


s,a) =p((k',l') | (k,l),a) = p{V | Z) p(k' \ (k,l), a ), 

(5) 


p(k' | ( k,l),a ) 



if k' = [k — p,(l,a)] + and a £ ^4^, 


otherwise, 


(6) 


3 For the detailed study of strategic network selection interactions among 
multiple MUs, we refer readers to our work in (2D. 

4 By allowing Hotspot 2.0 and ANDSF to complement with each other 0, 
(8), an MU can query for the speed and load in different types of networks 
ED before transmitting data in these networks. 


ipt(k,l,a) 

= ct(k,l,a ) + E E p({k',l')\(k,l),a)v t+1 (k',l') (9) 

I'GC k'CK 

= min{fc, p(l , a)}p(l, a) p{l'\l)v t +i(\k-p(l, a)]+ 1). (10) 

The first and second terms on the right hand side of (0 are the 
immediate cost and the expected future cost in the remaining 
time slots for choosing action a, respectively. The derivation 
of ( I I Oi l from 0 follows directly from 0. 0, and 0. For 
t = T + 1, we set the boundary condition as 

Vt+i{s) = CT+i(k,l) = h(k), VkeJC,l£C. (11) 

With the optimality equation, we are ready to propose 
the general DAWN algorithm in Algorithm Q] The algorithm 

5 Prototype systems, such as BreadCrumbs G3, can compute the movement 
probability by tracking the movement of the device’s owner. 

6 For simplicity, we assume that the total payment and penalty have equal 
weights. If we put a larger weight on the penalty than the total payment, then 
the probability of completing file transfer would increase, at the expense of 
an increase in the payment. 

7 Here, we use the term cost to represent both the payment in a at time 
f GT and the penalty in 0 at time T + 1. 
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Algorithm 1 General Delay-Aware Wi-Fi Offloading and 
Network Selection (DAWN) Algorithm. 

1: Planning Phase: 

2: Set vr+i(k,l), V k £ ZC, V l £ C using JT71 > 

3: Set t := T 

4: while t > 1 

5: for l £ £ 

6: Set k := 0 

7: while k < K 

8: Calculate i/jt(k,l,a), Va £ A^ using < 1 1 01 1 

9: Set Sf(k,l) := argmin {i/j t (k, l, a)} 

10: Set Vt(k, l) := 't/’t (fc, 1,6% (k,l)) 

11: Set k := k + a 

12: end while 

13: end for 

14: Set t t — 1 

15: end while 

16: Output the optimal policy 7r* for the transmission and Wi-Fi 
offloading phase 

17: Transmission and Wi-Fi Offloading Phase: 

18: Set t := 1 and k := K 

19: while t <T and k > 0 

20: Determine the location index l from GPS 

21: Set action a := Sf ( k, l ) based on the optimal policy tc* 

22: If a > 0 

23: Send p(l, 1) bits to the cellular network if a = 1 

or offload p(l, 2) bits to the Wi-Fi network if a = 2 
24: Set k := [k — p(l, a)] + 

25: end if 

26: Set t := t + 1 

27: end while 


consists of two phases, namely the planning phase and the 
transmission and Wi-Fi offloading phase. Let a > 0 be the 
granularity of the discrete state element k in the algorithm 
(such as 1 Mbits). First, in the planning phase, based on 
the optimality equation in 0 and the boundary condition in 
O . we obtain the optimal policy it* that solves problem 
using backward induction C3 pp.92]. Specifically, we 
first set VT+i(k,l) based on the boundary condition (line 2) 
of Algorithm Q] Then, we obtain the values of <5* (k.l) and 
vt{k : l) by updating them recursively backward from time slot 
t = T to time slot t = 1 (lines 3 to 16). Algorithm Q] has a 
computational complexity of O(KLTfo) JT4). 

Theorem 1: The policy tt* = (Sf (k, /), V k £ 1C, l g £,f £ 
T), where 

Sf(k,l) = argmin {tf) t (k,l,a)}, (12) 

a£A ( - l '> 

is the optimal solution of problem 0. 

Proof: Using the principle of optimality lf30l pp. 18], we 
can show that 7r* is the optimal solution of problem 0. ■ 

Notice that the optimal policy 7r* is a contingency plan that 
contains information about the optimal transmission decision 
at all the possible states ( k , Z) in any time slots t £ T, and the 
system computes it offline before the file transfer begins in the 
second phase. In the second phase, the MU first determines 
the location index l in each time slot based on the location 
information obtained by global positioning system (GPS) (line 
20). Then, the MU carries out the transmission decisions based 


on the optimal policy 7r* through checking a table (lines 
21 to 25), and updates the state element k accordingly (line 
24). As the complexity of Algorithm |T| is high in general, 
it motivates us to design an approximation algorithm with a 
lower computational complexity in the next section. 

VI. Threshold Policy and Monotone DAWN 
Algorithm 

In this section, we establish sufficient conditions under 
which the optimal policy has a threshold structure in the 
remaining file size k and time t. We then propose a monotone 
DAWN algorithm accordingly, which approximately solves 
problem 0 in the general case with a lower computational 
complexity. Thus, the results cannot be obtained by a direct 
application of the standard DP theory. 

Specifically, we make the following assumptions for deriv¬ 
ing the optimal policy in this section: 

Assumption 1: (a) The penalty function h(k) is convex 
and non-decreasing in k\ (b) Wi-Fi is free to the MU (i.e., 
p(l, 2) = 0, VZ £ £*41); (c) The cellular price is location- 
independent (i.e., p(l, 1) = p(V, 1), VZ, V £ £, l l'y, (d) 
The cellular and Wi-Fi data rates are location-independent 
(but these two rates are different in general). That is, pi = 
p(l, 1), VZ £ £ and p 2 = /x(Z,2), VZ £ £4); and (e) We 
approximate minjfc, p(l, 1)} in 0 by p(l, 1) for action a = 1. 

Notice that (a) a convex penalty function can be used to 
model the increasing marginal penalty for every additional 
unit of file segment not yet transferred. It is similar to the 
idea that a concave utility function can be used to model 
the diminishing marginal utility, (b) Free Wi-Fi can often be 
found in places such as homes, offices, or coffee shops, (c) 
Location-independent cellular price is widely used in practice, 
(d) is a good approximation when the cellular and Wi-Fi data 
rates across different locations have a small variance, (e) is 
a technical approximation for simplifying the structure of the 
optimal policy. 

With Assumption [Q the cost at state s with action a at time 
slot t is modified from 0 as 

c t (s,a)=ct(k,l,a)=I(a = l)q= < q ’ lf ” (13) 

I 0, otherwise, 

where a £ /(•) is the indicator function, and q = 

p(l, l)p(l, 1). As a result, i/jt(k,l,a) in @ can be rewritten 
as 

= I (a = 1 )q+ ^ p(l’\l)v t+1 ([k - p(l,a)} + ,1'). 

I'&C 

(14) 

A. Properties of the Optimal Policy 

First, we discuss some analytical results related to the 
properties of the optimal policy under Assumption |T] 

Lemma 1: (a) Vt(k,l) is a non-decreasing function in k, 
VZ £ £ T- (b) Vt(k,l) is a non-decreasing function in t, 

Vfc £ /C,Z £ £. 

The proof of Lemma Q] is given in Appendix lAl Intuitively, 
given a fixed location l £ £, the expected cost is higher when 
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k is larger (i.e., the remaining file size to transfer is larger) or 
when t is larger (i.e., it is closer to the deadline). 

Next, we characterize the optimal transmission policy at a 
location Z £ D- 1 ' with Wi-Fi. Since Wi-Fi is free for use. 
Lemma [2} a) states that action a = 2 (i.e., using Wi-Fi) is 
always preferred to action a = 0 (i.e., remaining idle). Lemma 
[2b) states that if the Wi-Fi data rate is higher than the cellular 
data rate, then the MU should always use Wi-Fi. 

Lemma 2: For any location Z £ (where Wi-Fi is 

available), we have: 

(a) ipt{k,l,0) > ipt(k, Z, 2), Vfc £ K.,t £ T. 

(b) If p(l , 1) < n(l, 2), then 5*(k, l) = 2, Vfc £ /C, t £ T. 

The proof of Lemma [2] is given in Appendix iBl Notice that 
at Z £ although A 1 ' 1 ' 1 = {0,1,2} from (0, Lemma 0a) 
implies that we do not need to consider action a = 0 in (0. 
Specifically, let 


A® = 



if Z £ £«, 
if l £ £(°>. 


(15) 


We can simplify the optimality equation in (0 


v t {k,l)= min {i/) t (k, l, a)} = min {ipt(k,l,a)}. (16) 

ae-AW a£i(‘) 


B. Threshold Structure of the Optimal Policy 

To show the threshold policy in dimension k, we need to 
leverage on the concepts of superadditivity and subadditivity 
lfl~5l pp. 103]. Specifically, with the assumptions we made on 
the penalty function and data rates, we show in Appendix [0 
that f> t {k,l,a) is superadditive or subadditive on 1C x A^ 
under different conditions. Then, with A} (k. I ) defined in ( 1 1 2b . 
we can establish the threshold structure of the optimal policy 
in dimension k El pp. 104, 115]. 

Definition 1: Given l £ C , the function ipt(k, Z, a) is super¬ 
additive on AC x A^ if for V k, k £ AC and V d, a £ A, where 
k > k and a > a, we have 

'ip t (k,l,a) +tp t (kj, a) > a) +tp t (k, l, a). (17) 

The function tp t {k,l,a) is subadditive on AC x A^ if the 
reverse inequality always holds. 

To prove the threshold policy in dimension t, we show 
in Appendix [E] that the incremental changes of v t (k.l) with 
respect to k is non-decreasing in time t. Overall, we state the 
threshold policy in both dimensions k and t as follows. 


Theorem 2: Under Assumption [I] the optimal policy 7r* = 
( 5%(k,l ), V/c £ AC, l £ £, t £ T) has a threshold structure in 
both k and t as follows: 

For location l £ C ! ' ,}> without Wi-Fi, we have 


6* t (kJ) 


1 (cellular), 
0 (idle), 


SKk,l) 


1 (cellular), 
0 (idle), 


if k > k*(l , t), 

- v vf £ r. 

otherwise, 

*<£*•(*,!), v|fceJc . 

otherwise. 


and 

(18) 

( 19 ) 


where k*(l,t) and t*(k,l) are location and time dependent 
thresholds in dimensions k and t, respectively. 

For location l £ C 1 ' with Wi-Fi, if the data rate of Wi-Fi 
is lower than that of cellular (i.e., p 2 < pf), we have 


W) = 


S* t (k,l) = 


I 1 (cellular), if k > k*(l,t ) 
|^2 (Wi-Fi), otherwise, 

1 (cellular), if t > t*(k,l), 


2 (Wi-Fi), otherwise, 
Otherwise (hence pi < p 2 ), we have 


V t £ T, and 
( 20 ) 

V k £ JC. 

( 21 ) 


5*(k,l) = 2 (Wi-Fi), V k £ AC, t £ T. (22) 

Theorem0states that when k is above a threshold (i.e., there 
are many bits waiting to be transmitted) or when t is above 
a threshold (i.e., the deadline is close), the MU should use 
the cellular network immediately to avoid the penalty (if Wi¬ 
Fi is not available or Wi-Fi is not fast enough). The proof of 
threshold policy in dimension k stated in ( [T8l > and < 12( )b is given 
in Appendix [D] The proof of threshold policy in dimension t, 
stated in (fl9] > and (f2TT> is given in Appendix [F] The result in 
(l22li is due to Lemma 0b). 

Furthermore, we use the threshold structure in Theorem 0 
to establish Theorem 0 which help to speed up the search of 
the thresholds. 

Theorem 3: (a) k*{l,t~ 1) > k*(l,t ), V/ £ £ TC 

(b) t*(k, l) > t*(k + cr,l), V l £ £, k £ AC. 

Basically, Theorem 0 states that the threshold in dimension 
k is non-increasing in t, while the threshold in dimension t 
is non-increasing in k. The proof of Theorem 0 is given in 
Appendix [0 


C. Monotone DAWN Algorithm 

With the threshold structure in both dimensions k and t from 
Theorems0and0 we propose Algorithm0with a much lower 
computational complexity than Algorithm 0 In Algorithm 0 
it should be noted that we choose to characterize the optimal 
policy 7r* using the thresholds (fc*(Z,f), VZ £ C, t £ T) in 
the file size dimension in (ITSl) and (l20l >. In the planning phase 
of Algorithm0 we use the procedure THRESHOLD to obtain 
the set of thresholds (fc*(Z,f), VZ £ C,t £ T) (line 10) in 
dimension k. Since we execute the algorithm backward from 
t = T to t = 1, after we have found the threshold k*(l,t ) at 
time t, we can reduce the search space of k*(l, t — 1) at time 
t — 1 by Theorem 0a). 

By knowing the threshold structure in both dimensions k 
and t, in Theorems 0 and 0 we can speed up the computation 
of the optimal policy. Let A th C A^ be the set of feasible 
actions that we should consider (procedure line 8) for the 
optimal policy. Instead of considering the two possible actions 
in = {1, 2} for Z £ £« and A® = {0,1} for Z £ £(°) 
in <fl5l> for (fl6l >. we can reduce the amount of computation by 
only considering one possible action in yl lh under two condi¬ 
tions: (i) When k < k*(l,t + 1), we know from Theorem0a) 
that k < k*(l, t), so we only need to consider ,4 th = {j} with 
one element (procedure line 2). (ii) When we have reached 
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Algorithm 2 Monotone DAWN Algorithm. 

1: Planning Phase (for 712 < 711 ): 

2 : Set vr+i{k, l), V k G JC,l G C using dTTT ) 

3: Set t~T 
4: while t > 1 
5: for l G £ 

6 : Call THRESHOLD procedure 

7: end for 

8 : Set t := t — 1 

9: end while 

10 : Output the thresholds ( k*(l , t), VI € £, t € T) for the transmis¬ 
sion and Wi-Fi offloading phase 
11 : Transmission and Wi-Fi Offloading Phase: 

12: Set t := 1 and A; := K 

13: while t < T and k > 0 

14: Determine the location index l from GPS 

15: If l G £ (0) 

16: If k > k*(l,t ), Set a := 1, else, Set a := 0, end if 

17: else if l G £^ 

18: If M 2 < Pi 

19: If k > k*(l, t), Set a := 1, else, Set a := 2, end if 

20 : else 

21 : Set a := 2 

22 : end if 

23: end if 

24: If a > 0 

25: Send ffll, 1) bits to the cellular network if a = 1 

or offload fi(l, 2) bits to the Wi-Fi network if a = 2 
26: Set k := [k — p,(l, a)] + 

27: end if 

28: Set t t + 1 

29: end while 


procedure THRESHOLD 

1: If / G £ (0) , Set j := 0, else. Set j := 2, end if 
2: Set _4 th := {j}, k := 0, and flag := 0 

3: while k < K 

4: if k > k* (l, t + 1) and flag = 0 

5: Set A th := {j. 1} and flag ~ 1 

6: end if 

7: Calculate if>t(k,l,a ), Va G A th using d 1 4I > 

8: Set 5t (k, l) := arg mm{tl>t(k, l, a)} 

aeA a 

9: Set v t (k,l) ■— ipt(k, l, 5*(fe, ()) 

10: if St ( k , l) = 1 and flag = 1 

11: Set A th := {1}, k*(l,t ) := k, and flag := 2 

12: end if 

13: Set k := k + a 

14: end while 


the threshold that k > k*(l,t ), we know from Theorem [2] 
that we only need to consider A th = {1} with one element 
(procedure line 11). In both cases (i) and (ii), A th becomes 
a singleton, and the minimization in line 8 of the procedure 
is readily known. As a result, the computational complexity 
is reduced from 0[KLT / a) in Algorithm Q] to approximately 
0(Lm&x{K/a,T}) in Algorithmic lfj~4) . In the second phase, 
we determine action a based on the threshold optimal policy 
in dimension k stated in Theorem[2] Specifically, the decisions 
in lines 16, 19, and 21 are due to ©, ( l20l >. and (l22l >. 
respectively. 


VII. Performance Evaluations 

In this section, we evaluate the performance of the general 
and monotone DAWN schemes by comparing them with three 
benchmark schemes (the no offloading, on-the-spot offloading 
M, and Wiffler (1 schemes) in terms of the total cost, 
probability of completing file transfer, and the total payment. 
We also illustrate the threshold policy stated in Theorem [2] 

For each set of system parameter choices, we run the 
simulations 1000 times with randomized Wi-Fi locations, data 
rates in the cellular and Wi-Fi networks, and the user mobility 
trajectories, and show the average value. The MU is moving 
within L = 16 possible locations in a four by four grid 
(similar to that in Fig. |T|). To generate the trajectory of the MU, 
we consider the state transition probabilities p(l' \ l), where 
we assume that probability that the MU stays at a location 
between two consecutive time slots is p(l \ l) = 0.6, W G C. 
Moreover, it is equally likely for the MU to move to any one 
of the neighbouring locations. As an example in Fig. [Q at 
location 7, the probability that the MU will move to one of 
the locations 3, 6, 8, or 11 is equal to (1 — 0.6)/4 = 0.1. For 
another example, at location 1, the probability that the MU 
will move to one of the neighbouring locations 2 and 5 is 
equal to (1 — 0.6)/2 = 0.2. 

Unless specified otherwise, we assume that the cellular data 
rate /x(2,l),VZ G C and the Wi-Fi data rate /u(Z, 2), VZ G C ^ 
are truncated (on the range of [0, 00 )) normally distributed 
random variables with means p c and p w , respectively, and 
standard deviations equal to 5 Mbps. We assume that the 
cellular usage price p(l, 1), V/ G £ is US $6/Gbyte, while 
the Wi-Fi is free such that p(l, 2) = 0, V/ G The 

probability that a Wi-Fi connection is available at a particular 
location is 0.5. The length of a time slot Af equals to ten 
seconds. We consider that the MU is transferring a file (e.g., a 
movie), where the deadline of the file transfer is D minutes (so 
T = 60D/At). We set the file size granularity a = 10 Mbits. 
For the delay violation penalty, we use the convex function 

h(k) = bk 2 : VhGlC, (23) 

where b > 0 is a constant. As an example, we adopt b = 1 in 
our simulations. 

Next we explain the five schemes in our simulations. Under 
the general DAWN scheme, we run the planning phase in 
Algorithm [F] with the complete and accurate data rate in¬ 
formation /z(Z, 1), VZ G C and /z(Z,2),VZ G For the 

monotone DAWN algorithm, however, we assume that the MU 
only knows the mean data rates in the networks. As a result, 
we run the planning phase in Algorithm [2] with incomplete 
data rate information by letting p(lA) = He, V l G C and 
p(l,2) = p w ,Vl G £W. Under the no offloading scheme, 
the MU uses the cellular network at all times. For the on-the- 
spot offloading (OTSO) scheme, the data traffic is offloaded to 
the Wi-Fi network whenever Wi-Fi is available. The MU will 
use the cellular connection immediately when Wi-Fi is not 
available. The Wiffler scheme is a prediction-based offloading 
scheme proposed in Fet ( be the estimated amount of 
data that can be transferred using Wi-Fi by the deadline. The 
Wiffler system uses a history-based predictor, which estimates 
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Fig. 2. The probability of completing file transfer versus deadline D for 
p(l 1 1) = 0.6, V l G C, K = 750 Mbytes, p c = 90 Mbps, and = 20 
Mbps. 



Fig. 3. The probability of completing file transfer versus average Wi-Fi data 
rate fi w for p(l \ l) = 0.6, V l € C, K = 625 Mbytes, deadline D = 1 min, 
and p c = 90 Mbps. 

£ based on the inter-meeting time and throughput of the last 
m Wi-Fi AP encounters. If Wi-Fi is available in the current 
location, then Wi-Fi will be used immediately. If Wi-Fi is 
not available, the MU needs to check whether the condition 
£ > 9k is satisfied. Here, k is the remaining size of the file to 
be transferred, and 6 > 0 is the conservative coefficient that 
tradeoffs the amount of data offloaded with the completion 
time of the file transfer. If this condition is satisfied, meaning 
that the estimated data transfer using Wi-Fi is large enough, 
then the MU will stay idle and wait for the Wi-Fi connection. 
Otherwise, the MU will use the cellular connection. We set 
9=1 and m = 4 as suggested in 0. 

A. Comparisons Among Different Schemes 

In this subsection, we compare the performance of the five 
schemes (two proposed in this paper and three benchmark 
schemes) under stringent and non-stringent deadline require¬ 
ments. First, we consider a larger file size K = 750 Mbytes, 
which is challenging to complete the transmission when the 
deadline is short. Here, we first focus on the special case, 
where the cellular data rates are much higher than the Wi¬ 
Fi data rates. Specifically, we consider that the mean cellular 
and Wi-Fi data rates are p c = 90 Mbps and p w = 20 Mbps, 


Fig. 4. Total cost versus deadline D for p(l 1 1) = 0.6, V l € C, K = 750 
Mbytes, p c = 90 Mbps, and p w = 20 Mbps. 


respectively, which are reasonable parameters under a 4G LTE- 
A cellular system ED and a congested Wi-Fi network l32l . 
Our later simulation results will show how the performance of 
the algorithms changes with different Wi-Fi data rates. In Fig. 
13 we plot the probability of completing file transfer against 
deadline D. As D increases, it is more likely to finish the file 
transfer before the deadline, so the probability of completing 
file transfer of the five schemes increases. Moreover, we 
observe that the general DAWN and no offloading schemes 
achieve the highest probability of completing file transfer, 
and the monotone DAWN scheme achieves a slightly lower 
probability. On the other hand, we observe that the OTSO 
and Wiffler schemes are not able to complete the file transfer 
around 40% of time when D = 2 mins. The reason is that 
these two schemes always offload the traffic to the Wi-Fi 
networks whenever Wi-Fi is available. However, they ignore 
the QoS requirement of the application in terms of the stringent 
deadline. When the cellular data rate is higher than the Wi-Fi 
data rate, it may be preferable to use the cellular network to 
increase the chance of file transfer completion despite of the 
higher payment. 

Since the result in Fig. [2] depends on the relative data rates 
in the cellular and Wi-Fi networks, we evaluate the probability 
of completing file transfer against the average Wi-Fi data rate 
//,,, under fixed average cellular data rate p c = 90 Mbps for 
K = 625 Mbytes and deadline D = 1 min in Fig. 0 As we 
can see, when //,,, increases, the probability of completing file 
transfer of the OTSO scheme approaches to that of the general 
DAWN scheme. It is because when the Wi-Fi data rate is much 
higher than the cellular data rate, OTSO becomes the optimal 
offloading decision, as Wi-Fi networks are free and have a 
higher data rate than the cellular network. 

In Fig. Q] we plot the total cost (i.e., the objective function 
in problem 0) against the deadline D for K = 750 Mbytes. 
Since the general DAWN scheme computes and obtains the 
optimal policy, it achieves the minimal total cost as stated in 
TheoremQ] Moreover, we observe that the total cost decreases 
with D for most of the schemes. The reason is that as D 
increases, the MU has more time to wait for the availability of 
free Wi-Fi, and thus reduces the total payment. Moreover, for 
a larger D, the chance of completing the file transfer is higher. 
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Fig. 5. The total data usage payment of the user versus deadline D for 
p(l | /) = 0.6, V l G C, K = 92.5 Mbytes, /x c = 90 Mbps, and fi w = 20 
Mbps. 



Deadline D (minutes) 


Fig. 6. The average completion time versus deadline D for K = 92.5 
Mbytes, p c = 90 Mbps, and p w = 20 Mbps. 


and the penalty is thus smaller. For the monotone DAWN 
scheme, however, we observe a slight increase in the total 
cost at D = 3 mins, which is probably due to the incomplete 
data rate information described above. As shown in Fig. [4] the 
general DAWN has a lower total cost than the no offloading 
scheme, which implies that the general DAWN requires a 
lower total payment to achieve the same highest probability 
of completing file transfer as the no offloading scheme does 
illustrated in Fig. [2] 

Next, we consider the case with a non-stringent deadline 
requirement due to a smaller file size K = 92.5 Mbytes, where 
all the schemes have a very high probability of completing 
the file transfer in this setup. In Fig. 0 we plot the total 
payment against deadline D under the five schemes. For the 
no offloading scheme, since it always uses the more expensive 
cellular network, its payment is the highest and is independent 
of D. For the OTSO scheme, it has a lower payment than 
the no offloading scheme, because it uses the free Wi-Fi 
networks whenever they are available. However, the OTSO 
scheme is not aware of the deadline, so it often incurs a 
significant penalty for violating the deadline. In contrast, the 
general DAWN, monotone DAWN, and Wiffler schemes are 
deadline-aware, where they evaluate the chance of file transfer 
completion by the deadline. When D increases, these three 
schemes use the Wi-Fi network more often to complete the 
file transfer, so the total payment decreases. In Fig. 0 we 


Fig. 7. The probability of completing file transfer versus deadline D for 
p(l 1 1) = 0.1, V l G C, K = 750 Mbytes, p c = 90 Mbps, and p w = 20 
Mbps. 



Fig. 8. The probability of completing file transfer versus average Wi-Fi data 
rate //,,, for p(l \ l) = 0.1, Vi 6 C, K •«■ 625 Mbytes, deadline D = 1 min, 
and p, c = 90 Mbps. 

observe that the monotone DAWN achieves the same lowest 
payment as the general DAWN. 

We study in more details on how time is spent before 
completing the file transfer under a non-stringent deadline 
requirement as in the setup in Fig. 0 In Fig. [6j we plot the 
average completion time of the five schemes for K = 92.5 
Mbytes under deadlines D = 2 mins and D = 5 mins. 
Notice that the completion time includes three parts: cellular 
transmission time (blue), Wi-Fi transmission time (green), and 
waiting time for Wi-Fi networks (brown). We can see that the 
no offloading and OTSO schemes have the shortest lengths 
of completion time, because of the zero waiting time. On the 
other hand, the Wiffler, monotone DAWN, and general DAWN 
schemes experience longer lengths of completion time due to 
the more significant waiting time. When the deadline is longer 
(D = 5 mins), these three schemes can tolerate a longer delay 
to wait for the availability of free Wi-Fi networks, and reduce 
their cellular transmissions, and thus their payments as shown 
in in Fig. [5] 

In Figs. ITlfTOl we run the simulation experiments in Figs. 
[20 again with different user movement probabilities, where 
the probability of staying at a location p(l 1 1) = 0.1, V/ £ C. 
We can see that the general trends of the curves and insights 
remain the same, although the magnitude of the performance 
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Fig. 9. Total cost versus deadline D for p{l \ l) = 0.1, VI G C, K =s 750 
Mbytes, p c = 90 Mbps, and fi w = 20 Mbps. 



Fig. 10. The total data usage payment of the user versus deadline D for 
p(l 1 1) = 0.1, V l G C, K = 92.5 Mbytes, fi c = 90 Mbps, and = 20 
Mbps. 

metrics differ. It suggests that the values of the movement 
probabilities do not have a significant impact on the insights 
of our simulation results. 

11. Demonstration of the Optimal Policy under Different 
Penalty Functions 

In addition, we illustrate the actions of the optimal policy 
for different system states with a simple example. We first 
look at the special case with convex penalty function h{k) and 
location-independent data rates ji \ and // 2 , and cost q — 1 in 
O for K = 20 Mbits, a = 1 Mbits, T = 20, and b = 10. In 
Figures [Tl(a)| and [ll(b)| we can observe the threshold structure 
in dimensions k and t as stated in ( fl~8l > and (03 for Z e £(°> 
and in < [20b and (EH for l £ with P 2 < pi as stated in 
Theorem [2] We can also notice the change in thresholds as 
stated in Theorem [3] 

Finally, we show an example of the optimal policy for 
the general case with non-convex penalty function h{k) and 
location-dependent cellular/Wi-Fi data rates. We consider a 
step penalty function h(k) = Z for k > 0 and h( 0) = 0, where 
Z >> 1 is a large positive constant. With this penalty function, 
the objective is to complete the file transfer with the minimal 
cellular usage. We adopt Z = 100000. As shown in Fig. QE] 
we can see that multiple thresholds exist along dimension k. 




Fig. 11. An example of the optimal policy at location l E C for the case with 
convex penalty and location-independent data rates, where K = 20 Mbits, 
a = 1 Mbits, T = 20, and b = 10. The white dots (o), black dots (•), 
and blue crosses (+) represent the transmission decisions of a = 0 (idle), 1 
(use cellular), and 2 (use Wi-Fi), respectively. We can observe the threshold 
optimal policy as stated in Theorem |2| and Theorem |3] 


instead of a single threshold in the special case. For example, 
in Fig. H2a), for t > 16, when k is increased from zero, 
the decision first changes from idle to using cellular, because 
a complete file transfer is still possible. However, when k is 
increased further that a complete file transfer is impossible, 
the idle action is chosen. Notice that it is very different from 
the policy in the special case as stated in Theorem 12 where 
the MU would not stay idle even when there is no chance 
to complete the file transfer. To sum up, the penalty function 
has a significant impact on the optimal policy, and it should 
be chosen carefully according to the QoS requirement of the 
application. 

VIII. Conclusions 

In this paper, we studied the user-initiated Wi-Fi offloading 
problem for delay-tolerant applications under usage-based 
pricing. The user aims to minimize its total data usage 
payment, while taking into account the deadline of the file 
transfer. We first proposed a general DAWN algorithm for the 
general case using dynamic programming. We then established 
sufficient conditions under which the optimal policy has a 
threshold structure in both dimensions k and /. As a result, 
we proposed a monotone DAWN algorithm with a lower 
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Fig. 12. An example of the optimal policy at location l E C for the case 
with step penalty and location-dependent data rates, where K — 20 Mbits, 
a = 1 Mbits, T = 20, and Z = 100000. The white dots (o), black dots (•), 
and blue crosses (+) represent the transmission decisions of a = 0 (idle), 1 
(use cellular), and 2 (use Wi-Fi), respectively. 


complexity that approximately solves the general offloading 
problem. It should be noted that the proposed algorithms are 
highly non-trivial, and they cannot be obtained simply by a 
standard application of dynamic programming. Contrary to 
the practices in some heuristic schemes that favour offloading 
traffic to Wi-Fi networks whenever possible, our simulation 
results showed that it is not always optimal for a user to 
perform Wi-Fi offloading when the deadline requirement is 
stringent and the data rate in the cellular network is much 
higher than that in the Wi-Fi network (e.g. a 4G LTE-A cellular 
system versus a congested Wi-Fi network). On the other hand, 
when the file transfer can be completed easily by the deadline, 
the delay-aware design in DAWN and Wiffler helps reduce the 
payment of the users. Overall, our results suggested that future 
cellular and Wi-Fi integration system should include dynamic 
offloading policies that take into account the users’ QoS and 
the real-time network loads, instead of using simplistic and 
static offloading policies. 

As we considered the user-initiated offloading, where users 
are usually self-interested, we focused on the offloading deci¬ 
sion of a single user. We believe that it is an important step 
towards a better understanding of the multi-user offloading 
problem. In fact, we have made a step forward by considering 
the interactions of the network selection and data offloading 


decisions of multiple users in l28l . However, in l28l . we 
assumed that the mobility trajectory of each MU is estimated 
accurately, and we did not consider the delay-tolerant appli¬ 
cation with a given deadline. In other words, it is not possible 
to use the approach in j28l to directly generalize the results 
in this paper to the multi-user case. 

In this work, we have focused on the single file transfer 
by a given deadline. For future work, we will consider the 
case of multiple file transfers at the same time, and solve the 
problem by dynamic programming with additional states and 
decision variables. Considering the challenges of analyzing 
the single file case as in this paper, obtaining closed-form 
analysis and low complexity heuristic with clear engineering 
insights will be very challenging. Moreover, in this paper, we 
consider Markovian user mobility model. It is an interesting 
direction for future research by considering other mobility 
models, especially the heavy-tailed distribution model ll33l . 
which has shown to be more accurate for modeling human 
mobilities. 
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Appendix 

A. Proof of Lemma [7] 

(a) We prove it by induction. First, from (fill . VT+i(k,l) = 
h(k ) is a non-decreasing function in k, V l £ C. Assume that 
Vt+i(k,l) is a non-decreasing function in fc, VZ £ £. From 
(ITlT i. since p(V \ l) > 0, VZ, V £ C and the function I(a = l)g 
is independent of k, ijt t (k,l,a) is a non-decreasing function 
in k, VZ £ C, a £ A. Thus, vt(k, l) in ([ 8 ]) is a non-decreasing 
function in k, VZ £ C. 

(b) We prove it by induction. First, for t = T, we have 

vr{k,l) = min {M{k,l, a)} < M(k,l, 0 ) 
aeAV') 

= \l)v T +i{k,l') = h(k) =v T +i(k,l). *' 24 ' ) 

z'e£ 

The first and second equalities are from © and dl4ll . and the 
last two equalities are from CD. Assume that vt+i(k,l) is 
a non-decreasing function in t, Vfc £ 1C, l £ C. From (IT4l) . 
since p(l'\l) > 0, VZ,Z' £ £ and the function I(a = l)g is 
independent of t, ipt(k, l, a) is a non-decreasing function in t, 
V k £ 1C, l £ C, a £ A. Thus, v t (k, l) in © is a non-decreasing 
function in t, V k £ 1C, I £ C. ■ 


B. Proof of Lemma \2\ 

Let k £ 1C and l £ C be given. 

(a) We have 

ipt(k, l, 0 ) = ^2 p(l' 1 1) v t +i(k, l') 

l ' ec (25) 

> £ p(Z' I l) v t+1 (\k - p(l, 2 )]+, V) = Mk , l, 2 ), 
fee 

where the two equalities are due to (IT4l) and the inequality is 
due to Lemma [7] 

(b) First, since p(l, 1) < p(l, 2), we have 


M k , l,l) = q+^2 p(l' 1 Z) v t +1 ([k - p(l, 1 )] + , l 1 ) 

l ' GC (26) 
> £ p(l' 1Z) v t+1 ([k - p(l, 2)] + , Z') = l, 2), 
i'ec 

where the two equalities are due to (IT4l) and the inequality is 
due to Lemma Q] Combining the results from (l25l > and (f26l> . 
from (fbH i. we have (k,l) = 2, V k £ K,,t £ T. ■ 


C. Superadditivity and Subadditivity of ipt(k, l, a) 

The proof of the threshold structure in dimension k in 
Theorem [2] is based on the results in Lemmas [3 and [4j Let 
l £ C be given. Let = {j, 1}, where j = 0 if / £ £(°) 
and j = 2 if l £ as in 03, and /io = 0. With only two 
possible actions in A^ l \ we can rewrite CD as 


Mk, l,a) = I(a = 1 )q + Y] p(l' \ l) I(a = 1 ) 


e£ 


xv t +i([k- pi} + , l’)+ (l- I(a = l))v t+ i([k - pj] + ,l') . 

(27) 


Lemma 3: If pj < p\ and h(k) is a convex and non¬ 
decreasing function in k, then 

v t ([k - Pj] + ,l) - v t ([k - pi) + ,l) > v t ([k - a - pf + ,l) 
-v t ([k-a-p 1 ] + ,l),Vk £ 1C, l £ C,t £ TU{T+1}. 

(28) 

Proof: We prove it by induction. Since h(k) is a non¬ 
decreasing convex function, we have 

M [k - M + ) ~ h ([k ~ Mi] + ) > h([k - a - pf + ) 

—h([k — a — /ii] + ), VZc£/C. 

Let /c £ ZC, Z £ £ be given. For t = T + 1, we have 
vt+i([ k - Pj] + ,l) -v T +i([k - pi} + ,l) 

= h([k - pj} + ) - h([k - pi} + ) 

> h([k — a — Pj] + ) — h([k — a — /xi] + ) 

= v T +i([k-a - Pj] + ,l) - v T +i ([k - a - pi] + J), (30) 

where the equalities are due to ( ITTb and the inequality is due 
to ( l29l ). Assume that for a given t £ T, we have 

v t +i{\k-Pj\\l)-Vt+i([k-pi\\l) >v t +i([k - a - Pj]\l) 
—Vt+i ([fc — a — £ K.,1 £ C. 

(31) 

From ©. let actions a±, a 2 , 0.3, 04 £ be defined such 
that 

v t ([k - Pj] + ,l) = min {M[ k ~ Pj] + A, a)} 

= M[k ~ Hj] + A,a 1 ), (32) 

vt([k - pi] + ,l ) = min {i/j t ([k - pi ] + ,(,a)} 

= tpt([k - pi] + ,l,a 2 ), (33) 


v t ([k - a - Pj] + ,l) = min {ip t ([k - a - pj] + ,l, a)} 
aEAV) 

= 4> t ([k - a - pj] + ,l,a 3 ), and (34) 

v t ([k-<j-p i] + ,() = min {4> t ([k - a - /zi] + , l, a)} 
aeAW 

= M\k - a - ftt] + ,(,a 4 ). (35) 

We thus have 


v t ([k - Pj] + ,l) - v t ([k - pi] + ,l) 

- v t ([k - a - Pj] + ,l ) T v t ([k - a - pi] + ,l) 

= M[k ~ Pj} + A,a 1 ) - ip t ([k - pi] + , l, af) 

- ip t ([k - a - pj] + ,l,a 3 ) +tl> t ([k - a - pi} + ,l,af) 

= M\k - Pj] + A,cli) -i/t t ([k - a - /i i ] + ,(,a 1 ) 

'--- . -' 

A 


+ ipt([k - cr - pj] + , l, ai) - 4> t ([k - a - Pj] + ,l, a3) 

'- . --- 

B 

(~M\k - Pi] + Aa 2 ) +tpt([k - pi ] + ,(,a 4 )) 

^-v- 

c 


- (ih([k - pi\ + - ipt([k - a - p{\ + ,/,a 4 )^) 


D 


= A + B + C — D. 


(36) 
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We have 

A = J2 p ( 1 ' I 0 J ( ai = 1 )[ y t+i (i k - N - H + >0 

vec 

- v t+ i ([k -a - Hj - p,i] + , Z')] + (l - I(ai = 1)) 
x [v t+ i([k - 2 Hj] + ,V) - v t+ i{[k - £7 - 2^] + ,/')] 

> Y p ( l> \ l )[ v t+i(i k - N - mi ] + 4 ') 

I'GC 

- v t+ i([k - a - fij - m} + , i')] 

> Y p(l' I 0 T(a 4 = 1) [«i+l([fc - 2yLfc 1 ] + , Z') 
vec 

- Vt+i([k - a - 2^i] + ,Z')] + (l - /(a4 = 1)) 

x^t+iQfc - m - Hi]V) - vt+i([k-<J-Hj-Hi] + i l ')\ 

= D, (37) 


where the two equalities are obtained by using (ITH and the 
two inequalities are obtained due to the induction hypothesis 
in (OH . From (l34l) and OH) , we have B > 0 and C > 0, 
respectively. Overall, from ( l36l >. we obtain 

v t ([k - Hj} + , 0 ^ v t ([k - /ii] + , l) -v t ([k-a - Hj} + , l) 
+v t ([k - a - Hi] + J) > 0, 

(38) 


which completes the proof. 


condition of Theorem [2] is satisfied, by iteratively applying 
Lemma [3] we have 

v t {[k - Hj] + , Z) - vt([k - /ri] + , l) 

> vt ([k - a - Hj] + , l) - v t ([k - a - ^ti] + , ()>••• 

> v t ([k - za - Hj] + J) - v t ([k - za- Hi} + J) 

= v t ([k - Hj] + J) - v t ([k - Mi] + ,/). (41) 

For l £ we consider j = 0 (see Appendix let. Since 
0 = Ho < Hi’ ipt(k,l,a) is subadditive on 1C x A^ from 
LemmalU From (131 pp. 104,115], <5^(fc,Z) is a monotone non¬ 
decreasing function in k. From (ITH and ( ITH . since 6%(k,l) £ 
„4<3) = {0, 1}, df(k, l ) is in the form of dTH . 

Then, we consider l £ £W for H 2 < Hi- Since j = 2 (see 
Appendix let, ip t (k,l,a) is superadditive on AC x A^ from 
LemmalU From 115] pp. 104,115], S*(k, l) is a monotone non¬ 
increasing function in k. From ( ITH and (ITH . as S^(k,l) £ 
J[(0 = {1, 2}, df(k, l) is in the form of (l20l) . ■ 


E. Incremental Changes ofi>t(k,l) 

The proof of the threshold structure in dimension t in 
Theorem [2] is based on the results in Lemmas [5] and [6] which 
establish that the incremental changes of vt(k,l) with respect 
to k is non-decreasing in time t. 

Lemma 5: If h(k) is a convex and non-decreasing function 
in k, then we have 


Lemma 4: If Hj < Hi an d V k, k £ AC, l £ C, t £ T with 
k > k, where 

v t +i([k - Hj] + J) - v t +i([k - Hi] + J) (39) 

> v t +i([k - Hj] + J) -vt+i([k- Hi] + J)i 

then if> t (k,l,a ) is subadditive on AC x A^ for j = 0, and 
superadditive on AC x A^ for j = 2, Vf £ T, respectively. 

Proof: Let k,k £ AC, d, a £ A^ l \ l £ £, and i £ T be 
given, where k > k and a > a. Then 


ipt(k,l, a) + ipt(k,l,a) - ij)t(k, l, a) - tpt{k,l , a) 

|Z)(/(a = 1) - I (a = 1)) v t +i([k - Hj] + J)' 


i'ec 


v t +i ([k~Hi] + , l) - v t +i([k-Hj} + , l ) + v t+ i ([fc-/ri] + , l) 


(40) 


where the equality is derived using ( 1271 ). Notice that p{V \ l) > 
0, £ C. First, for j = 0, we have a, a £ {0,1}, so 

J(a = l)</(a = l). From the given condition in LemmalU 
and Definition!]] we conclude that i/jt{k, l, a) is subadditive on 
AC x A.( l \ On the other hand, for j = 2, we have a, a £ {1, 2}, 
so /(a = 1) > /(a = 1). We can then conclude that f)t(k, l, a) 
is superadditive on AC x A^ l \ ■ 


D. Proof of Threshold Policy in Dimension k in Theorem \7} 

We consider the case 0 < Hj < Pi- Let k, k £ AC, l £ C , 
and t, £ T be given. Let k = [k — zo] + , where 2 > 0. If the 


v T +i([k - Hj] + J) - v T +i([k - Hi] + J) > vr{[k - Hj] + J) 

-v T {[k- Hi} + J), V/c £ K .,1 £ £. 

(42) 

Proof: First, by CD, we have 

LHS = vt+i ([k - Hj] + 7 0 - vt +l ([k - Hi] + J) 

= h([k - Hj ] + ) - K\k - Pi} + )- 
Next, we obtain 
RHS 

=vr{[k - Hj] + 1 0 - v T ([k - Hi\ + ,1) 

= min {il> T {[k - Hj) + ,l,j),ipT([k - Hj] + ,l,l)} 

- min{^T([fc ^ Hi] + ,hj), ^T<,[k - Hi} + 7 1, 1)} 

= min | P( 1 ' I 0 v T +i{[k - 2 Hj] + ,l'), 

I'ec 

4+ Y p ( l '\ l) VT +i{\k~ Hi -Mi] + 4')} 
i'e£ 

- min | Y P( 1 ' I 0 y T +1 ([k - Hj ~ Hi] + 7 I'), 

q+ ^p(('|0vT+i([fc-2/n] + ,(')| 

I'eC 

= min jh([fc - 2 Hj] + ), 9 + h([k - Hj ~ £i] + )} 

- min|/i([fc - Hj ~ £i] + ) ; 9 + h{[k - 2/ri] + ) j. (44) 

The second, third, and fourth equalities are due to ({s}- (HU), 
and CD- respectively. We consider the following two cases: 
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Case I: q + h([k — Pj — /ri] + ) > h([k — 2 pj] + ). In this case, 
we have 


q > h([k - 2 pj] + ) - h([k - pj - pi} + ) 
> h([k - pj - /ti] + ) - h([k - 2 p 7 } + ), 


where the second inequality is due to the fact that h(k) is a 
convex and non-decreasing function in k, and pj < p\ . Thus, 
we obtain q + h([k — 2 pi\ + ) > h([k — pj — pi\ + ). As a result, 
we have 


RHS = h([k — 2 Pj] + ) — h([k — pj — pi} + ) 

< h([k — Pj] + ) — h([k — /ti] + ) = LHS, 


(46) 


where the inequality is established for convex and non¬ 
decreasing h(k) and pj < p\. 

Case II: q + h([k — pj — p i] + ) < h([k — 2pf] + ) In this case, 
we have 

RHS = q + h([k — fij — /^i] + ) 

-min{/i([fc - pj - pi] + ),q + h([k - 2 /ii] + )j. ^ 

We consider the following two subcases in Case II: (a) First, 
if h([k — fij — ^i] + ) < q + h([k — 2p\] + ), then we have 

RHS = q + h([k — pj — pi] + ) — h([k — /.ij — pi] + ) 

< h([k - 2fj,j] + ) - h([k - pj - /^i] + ) (48) 

< h([k — Pj] + ) — h([k — /^i] + ) = LHS, 

where the first inequality is due to the given condition in Case 
II, and the second inequality is due to the convex and non¬ 
decreasing h(k). 

(b) Second, if h([k — Pj — p\] + ) > q + h([k — 2pi] + ), then 
we have 

RHS = h([k - pj - pi] + ) - h([k - 2p 1 } + ) 

< h([k — Pj] + ) — h([k — a*i] + ) = LHS, 

where the inequality is due to the fact that h(k) is a convex 
and non-decreasing function in k. Combining all the cases in 
©. and (gg, we have LHS > RHS. " ■ 

Lemma 6: If h(k) is a convex and non-decreasing function 
in k, then we have 

v t +i([k - Pj] + ,l) - v t +i([k - /ti] + , l) > v t ([k - Pj] + ,l) 
-v t ([k-p i]+,Z), V k g 1C,l S C,t gT. 

(50) 

Proof: We prove it by induction. First, from Lemma [5] 
we have established the result for t = T. Assume that for a 
given t G T, we have 

v t + 2 {[k - Pj] + ,l ) - v t+ 2 {[k - pi) + ,l) 

> v t +i([k - Pj) + ,l) - v t +i([k - pi) + ,l),Vk €lC,l e C. 

(51) 


Let actions 05 , a 8 , 07 , as £ be defined such that 


v t +i([k - Pj] + ,l) = min {ip t+1 ([k - Pj] + ,l, a)} 

= i/) t +i([k - Pj] + ,l,a 5 ), (52) 


v t +i([k - pi] + ,l) = min {f t+l ([k - p 7 ] + ,l,a)} 
ctEAW 

= t/jt + i([k-m} + ,l,a 6 ), (53) 


v t ([k - Hj] + , 1 ) = min {ip t ([k - l, a)} 

aeAW 

= ipt{[k - Hj] + ,l,a 7 ), and (54) 


v t ([k~n i] + ,() = min {i/> t ([k - /zi] + , l, a)} 

aEA^ 

= ip t ([k - ni] + ,l,a$). (55) 


We thus have 


v t +i ([k - Pj] + ,l) - v t +1 ([k - /ri] + , l) 

- v t ([k - +v t ([k -/ii] + ,Z) 

= il>t+i([k - Pj} + , l, o 5 ) - ^t+i([k - pi ] + ,(,ae) 

- i>t{[k - pj] + ,l,a 7 ) + ip t ([k - pi] + ,l,a 8 ) 

= ip t+ i([k - pj] + ,l,a 5 ) - ip t ([k - pj] + ,l,a 5 ) 

^-------^ 

E 

+ ipt{[k - pj} + ,l,a 5 ) — ijjt([k - Pj] + ,l,a 7 ) 

S ----^ 


$t+i([k - pi} + ,l,a 6 ) +i/j t +i ([k - pi ] + J,a s )^j 

"-v-- 

G 

- (ipt+i ([fc - pi] + ,l,a 8 ) - ip t ([k - /ri] + ,i,a 8 )^) 

H 


= E + F + G - H. 


(56) 


We have 

E = '^2p(l'\l) I(a 5 = l)[v t + 2 {[k - Pj - Pi] + ,l') 

I'EC 

-v t+ i{[k - pj - pi] + ,l')] + (l - I(a 5 = 1)) 
x [v t+2 {[k - 2pj] + ,1') - v t+ i{[k - 2p J ) + 1 l')] 

> p( 1 ' I 0 [vt+ 2 ([k - Pj - Pl) + , l') 

I'EC 

— Vt+i([k — Pj -^i] + ,0] 

> 5^p(i'IO I ( a s = l)[vt+ 2 {[k- 2 p 1 ] + ,l') 

I'EC 

- V t+ i([k - 2p 1 ] + 1 l')\ + (1 - I (as = 1 )) 
x[v t + 2 ([k-pj - pi] + , l') -v t+ i([k- Pj - p!} + , l')] 

= H, (57) 


where the two equalities are obtained by using (ITTI) and the 
two inequalities are due to the induction hypothesis in (BTl) . 
From (f54b and ([531) . we have F > 0 and G > 0, respectively. 
Overall, from d56l) . we obtain 


v t +i([k - Pj] + ,l ) - v t +i([k - pi] + j) 

> v t ([k - Pj] + ,l) - v t ([k - pi] + ,l), 


( 58 ) 


which completes the proof. 
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F. Proof of Threshold Policy in Dimension t in Theorem \2} 

Assume that there exists t £ T such that ipt(k,l, 1) < 
ipt(k,l,j). In this way, we have S^(k,l) = 1 from (fl2li and 


v t +i ([k-fj,j] + ,l') 
vec 

10 v t+2 ([k ~ Fj] + ,l') 


i’ec 


- v t +i([k - ni\ + ,l') 


-v t + 2 ([k-fji] + ,l') , 
(59) 


where the first inequality is by the definition in (Thil l. and 
the second inequality is from Lemma [6] It implies that 
< ipt+i(k, l,j), so 6t + i(k,l) = 1 from QUl. 
Overall, we show that if there exists t £ T such that 
5t(k,l) = 1, then 5* +1 (k,l) = 1, which establishes the 
threshold structure of the optimal policy in the time dimension. 


G. Proof of Theorem [T] 

Let j = 0 for l £ C (°) and j = 2 for l £ C ^ as mentioned 
in Appendix O 

(a) Let l £ C and t £ T be given. By the definition of 

threshold k*(l,t) in (f]~8l > and (f20t . we have 5t(k,l) = j for 
0 < k < k*(l,t). From the threshold structure in time in (IThl i 
and (fTil) . it implies that l) = j for 0 < k < k*(l,t). 

By the definition of threshold k*(l,t — 1) at time t — 1, we 
can conclude that k*(l,t — 1) > k*(l,t). 

(b) Let l £ C and k £ l C be given. By the definition of 

threshold t*(k,l) in (fl9l > and dTfl . we have Sf(k,l) = 1 for 
t > t*(k,l). It implies that S^(k + a, l) = 1 for t > t*(k,l) 
due to the threshold structure in file size in (QjU and (120b . By 
the definition of threshold t*(k + cr,l) for file size k + er, we 
can conclude that t* (k, l) > t* (k + a, l). ■ 






